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LISTING OF THE CLAIMS 

1 . (Original) A method of evaluating the quality of voice input recognition by a voice 
portal, said method comprising the steps of: 

extracting a current grammar from the voice portal; 

generating a test input for the current grammar, the test input including a test pattern and 
a set of active grammars for the current grammar; 
providing the test input to the voice portal; 

analyzing the test pattern with respect to the set of active grammars with a speech 
recognition engine in the voice portal; and 

deriving a measure of quality of recognition for the current grammar. 

2. (Original) The method of claim 1, wherein said deriving step includes the step of 
deriving a confidence level and a set of n-best results for the test input, and further comprising 
the steps of: 

comparing the confidence level and set of n-best results for the test input with an 
expected value to assess the measure of quality of recognition. 

3. (Original) The method of claim 1, further comprising the steps of: 

modifying the current grammar to create a modified grammar if the measure of quality of 
recognition for the current grammar deviates from a pre-determined range. 
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4. (Original) The method of claim 3, further comprising the steps of: 

(i) generating a test input for the modified grammar, the test input including a test pattern 
and a set of active grammars for the modified grammar; 

(ii) providing the test input for the modified grammar to the voice portal; 

(iii) analyzing the test pattern for the modified grammar with respect to the set of active 
grammars corresponding to the modified grammar with the speech recognition engine in the 
voice portal; 

(iv) deriving a measure of quality of recognition of the modified grammar; and 

(v) re-modifying the modified grammar and repeating steps (i) through (iv) until the 
measure of quality of recognition of the modified grammar does not deviate from a pre- 
determined range. 

5. (Original) The method of claim 1, further comprising the steps of: 

modifying the test pattern to emulate one or more user voices prior to entering the test 
input into the voice portal. 

6. (Original) The method of claim 1, further comprising the steps of: 
modifying the test pattern to emulate the influence of one or more communications 

network qualities prior to entering the test input into the voice portal. 

7. (Previously Amended) A computer program product for evaluating the quality of 
voice input recognition by a voice portal, said computer program product comprising computer 
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usable program code including a routine set of instructions which when executed by a machine 
cause the machine to perform the steps of: 

extracting a current grammar from the voice portal; 

generating a test input for the current grammar, the test input including a test pattern and 
a set of active grammars for the current grammar; 
providing the test input to the voice portal; 

analyzing the test pattern with respect to the set of active grammars with a speech 
recognition engine in the voice portal; and 

deriving a measure of quality of recognition for the current grammar. 

8. (Currently Amended) The machine readable storage computer program product of claim 
7, wherein said deriving step includes the step of deriving a confidence level and a set of n-best 
results for the test input, and further causing said machine to perform the steps of: 

comparing the confidence level and set of n-best results for the test input with an 
expected value to assess the measure of quality of recognition. 

9. (Currently Amended) The machine readable storage computer program product of claim 
7, further causing said machine to perform the steps of: 

modifying the current grammar to create a modified grammar if the measure of quality of 
recognition for the current grammar deviates from a pre-determined range. 

10. (Currently Amended) The machine - readable storage computer program product of claim 
9, further causing said machine to perform the steps of: 



4 



Application No. 10/733,995 
Filed: 12/11/2003 

Attorney Docket No.: BOC920030102US1 (1082-005U) 

(i) generating a test input for the modified grammar, the test input including a test pattern 
and a set of active grammars for the modified grammar; 

(ii) providing the test input for the modified grammar to the voice portal; 

(iii) analyzing the test pattern for the modified grammar with respect to the set of active 
grammars corresponding to the modified grammar with the speech recognition engine in the 
voice portal; 

(iv) deriving a measure of quality of recognition of the modified grammar; and 

(v) re-modifying the modified grammar and repeating steps (i) through (iv) until the 
measure of quality of recognition of the modified grammar does not deviate from a pre- 
determined range. 

1 1 . (Currently Amended) The machine roadablo storage computer program product of claim 
7, further causing said machine to perform the steps of: 

modifying the test pattern to emulate one or more user voices prior to entering the test 
input into the voice portal. 

12. (Currently Amended) The machine roadablo storage computer program product of claim 
7, causing said machine to perform the steps of: 

modifying the test pattern to emulate the influence of one or more communications 
network qualities prior to entering the test input into the voice portal. 

1 3 . (Original) A system for evaluating the quality of voice input recognition by a voice 
portal having a speech recognition engine, comprising: 
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an analysis interface for extracting a set of current grammars from the voice portal; 

a test pattern generator for generating a test input for each current grammar, the test input 
including a test pattern and a set of active grammars corresponding to each current grammar; 

a text-to-speech engine for entering each test pattern into the voice portal; 

a results collector for analyzing each test pattern entered into the voice portal with the 
speech recognition engine against the set of active grammars corresponding to the current 
grammar for said test pattern; and 

a results analyzer for deriving a set of statistics of a quality of recognition of each current 
grammar. 

14. (Original) The system of claim 13, wherein the set of statistics includes a confidence 
level and a set of n-best results for each test input, and wherein the results analyzer is configured 
to compare the confidence level and set of n-best results for each test input with an expected 
value to assess the quality of recognition of each current grammar having said test input with 
respect to its corresponding set of active grammars. 

15. (Original) The system of claim 13, wherein the test pattern generator is configured to 
modify each test pattern to emulate one or more user voices prior to entering the test input into 
the voice portal. 
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16. (Currently Amended) The system method of claim 13, further comprising the steps of: 

wherein the test pattern generator is configured to modify each test pattern to emulate the 

influence of one or more communications network qualities prior to entering the test input into 
the voice portal. 
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